ABSTRACT
Introduction Previously our group had identified 20 features which were associated with the development of upper gastrointestinal (UGI) cancers using a machine learning approach.[1] We sought to refine this model and to validate this in an independent dataset to assess its generalisability in an interim analysis. Methods We selected patients who were recruited for the multicentre Saliva to Predict rIsk of disease using Transcriptomics and epigenetics (SPIT) study to develop our model. Patients were recruited from 2-week wait suspected UGI pathways and additionally enriched with patients with confirmed oesophageal adenocarcinoma admitted as inpatients. We used regularised logistic regression (glmnet) from the caret package in R software to create the model. 60% of the data with 10-fold cross validation was used for training, with the remaining 40% for testing. For validation, we used data from the predicting RIsk of disease uSing detailed Questionnaires (RISQ) study, an ongoing prospective multicentre study using the questionnaire based on the our previous work.1 We evaluated the model using area under the receiver operating characteristic curve (AUC). Results We included 93 cancer and 715 non-cancer patients for training and testing and 21 cancer and 203 non-cancer patients for validation. We further reduced the model to 18 features without significant detriment to model performance. In the training and testing data AUC was 0.86 (95%CI: 0.81- 0.91) and 0.75 (95%CI: 0.67-0.83) respectively. We set a threshold of 0.03 as a cut off based on a cost function where false negatives had a 50-time greater impact than false positive cases (figure 1). For the validation cohort we achieved an AUC of 0.95 (95%CI: 0.90-1.00). This equated to a sensitivity 0.952 and a specificity of 0.897 for detecting cancer. Conclusions Initial results from our model compare favourably with the Edinburgh Dysphagia Scale, which has a sensitivity and specificity of 0.984 and 0.093 respectively.2 It also appears to have a high specificity, potentially helping to reduce unnecessary endoscopies. We aim to further increase the size of the validation cohort to ensure its robustness and generalisability. Our model could be applied to triaging and prioritising endoscopic referral backlogs as a result of COVID- 19.3.
ABSTRACT
Introduction Machine learning methods have been used to develop predictive models in gastroenterology.1 Previously we identified features including age, history of psychological disorders and severity of dysphagia symptoms which were correlated with upper gastrointestinal (UGI) cancers.2 We sought to create a machine learning based model which could be used to predict the presence of UGI in patients referred for endoscopy. Methods Patients were recruited as part of the Saliva to Predict rIsk of disease using Transcriptomics and epigenetics (SPIT) study. Patients were recruited from 2-week wait suspected UGI pathway referrals at 20 hospitals in the United Kingdom. We enriched the cohort with additional patients admitted with confirmed oesophageal adenocarcinoma. 60% of the data was used for model generation with 10-fold cross validation, while the models were tested on the remaining 40% of the data. We used seven methods to generate our models: Linear Discriminant Analysis (lda), Classification and Regression Tree (cart), k-Nearest Neighbour (knn), Support Vector Machines (svm), Random Forest (rf), Logistic Regression (glm) and Regularised Logistic Regression (glmnet). Model performance was assessed using area under the receiver operating characteristic curve (AUC) and DeLong test was used for model comparison. Results 93 cancer and 715 non-cancer patients were included. The best three models with 18 features were glmnet, lda and glm which all achieved an AUC of greater than 0.80 (figure 1). For the testing dataset, AUC was 0.75 (95%CI: 0.67- 0.83), 0.74 (95%CI: 0.66-0.82) and 0.75 (95%CI: 0.68-0.83) (p=ns for all 3 pairwise comparisons) respectively. When applying a cost function, the three models all achieved a sensitivity of 0.973 and a specificity of 0.234 to 0.388 for the testing dataset. Conclusions Our models compare favourably with the Edinburgh Dysphagia Scale, which has a sensitivity and specificity of 0.984 and 0.093 respectively.3 Our models have the advantage of an improved specificity, which could equate to fewer endoscopies being performed for low risk patients. Given rising waiting lists as a direct result of COVID-19, our tool could be used to prioritise patients who should be investigated sooner.4 We plan next to validate our models on a validation cohort to assess its generalisability.
ABSTRACT
Introduction Waiting times for endoscopy are rising rapidly following the COVID-19 pandemic. In addition, cancers may be missed as patients are placed on routine waiting lists but not monitored. Some hospitals use the Edinburgh Dysphagia Score to assess and prioritise patients for investigation. This offers a sensitivity of 98.4% and specificity of 9.3% to detect malignancy in patients presenting with dysphagia.4 However, it is not designed for detecting gastric cancer. We aimed to create a more accurate screening questionnaire as an aid to triaging referrals. Methods Patients were recruited as part of the Saliva to Predict rIsk of disease using Transcriptomics and epigenetics (SPIT) study. Patients were recruited from 2 week-wait suspected upper gastrointestinal cancer pathway referrals at 20 hospitals in the United Kingdom. The cohort was further enriched with patients found to have oesophageal adenocarcinoma on emergency hospital admission. They completed over 200 questions about a wide variety of symptoms and risk factors. After data cleaning, 800 patients were available for evaluation. Of these, 80 had upper GI cancer. A machine learning model was developed to identify those at highest risk of having upper GI cancer using a 'cost-based' approach which maximises the chance of detecting cancer. Information gain was followed by correlated feature selection and a multivariable logistic regression curve was created with scores from 0 (cancer very unlikely) to 100 (cancer very likely). The training dataset used 80% of the data and the model was tested with the other 20%. Results 20 features were found to be important and reproducible. They included age, sex, dysphagia, odynophagia, early satiety, weight loss, duration of chest pain and regurgitation, frequency of acid taste in the mouth, a previous history of smoking, cancer or psychological disorders, current anxiety level and frequency of vegetable intake. The area under the receiver operator curve to detect cancer was 0.83. 50% of cancers scored greater than 85 whereas 50% of normals scored less than 25. At a cut-off score of 10, sensitivity was 98.7% with specificity 26.8% to detect cancer (figure). Conclusions We have created a simple, reproducible risk score to identify patients at high and low risk of upper GI cancer. It performs better than previous scores but now needs testing in the real world. It might be usable to both upgrade routine patients to urgent endoscopy and remove patients at very low risk from waiting lists, thereby helping to prioritise patients with a greater clinical need and reducing the endoscopic backlog.
ABSTRACT
Introduction: Waiting times for endoscopy are rising rapidly following the COVID-19 pandemic, leading to significant backlogs.1 Modelling has demonstrated that delays in presentation to health services and delays in completing diagnostic procedures will lead to excess mortality.2 In addition, many cancers are likely to be missed as patients are placed on routine waiting lists but are not regularly monitored. Some hospitals use the Edinburgh Dysphagia Score to risk assess and prioritise patients for investigation.3 This offers a sensitivity of 98.4% and specificity of 9.3% to detect malignancy in patients presenting with dysphagia.4 However, it is primarily not designed for detecting gastric cancer. We aimed to create a more accurate screening questionnaire to risk assess patients and prioritise those who need early endoscopy. Aims & Methods: Patients were recruited as part of the Saliva to Predict rIsk of disease using Transcriptomics and epigenetics (SPIT) study. Ethical approval was gained from the Coventry and Warwickshire Regional Ethics Committee (17/WM/0079). Patients were recruited from 2 week-wait pathway referrals at 20 hospitals in the United Kingdom, which is used by physicians to refer patients who have may suspected cancer for further investigation The cohort was further enriched with patients found to have oesophageal adenocarcinoma on emergency hospital admission. They completed over 200 questions about a wide variety of symptoms and risk factors. After data cleaning, 800 patients were available for evaluation. Of these, 80 had upper GI cancer. A machine learning model was developed to identify those at highest risk of having upper GI cancer using a 'cost-based' approach which maximises the chance of detecting cancer. Information gain was followed by correlated feature selection and a multivariable logistic regression curve was created with scores from 0 (cancer very unlikely) to 100 (cancer very likely). The training dataset used 80% of the data and the model was tested with the other 20%. Results: 20 features were found to be important and reproducible. They included age, sex, dysphagia, odynophagia, early satiety, weight loss, duration of chest pain and regurgitation, frequency of acid taste in the mouth, a previous history of smoking, cancer or psychological disorders, current anxiety level and frequency of vegetable intake. The area under the receiver operator curve to detect cancer was 0.83. 50% of cancers scored greater than 85 whereas 50% of normals scored less than 25. At a cut-off score of 10, sensitivity was 98.7% with specificity 26.8% to detect cancer. Conclusion: We have created a simple, reproducible risk score to identify patients at high and low risk of upper GI cancer. It performs better than previous scores but now needs testing in the real world. It might be usable to both upgrade routine patients to urgent endoscopy and remove patients at very low risk from waiting lists, thereby helping to prioritise patients with a greater clinical need and reducing the endoscopic backlog.